Assistant Professor | School of Information Science
What is the PURPOSE of Quantitative Research?
Quantitative research:
What are the STEPS in Quantitative Research?
What are the characteristics of a good research problem?
Good research problems are grounded in empirical and theoretical literature, stated clearly, testable, feasible, and of vital interest to you
What is the difference between a conceptual definition and an operational definition?
These are not interchangeable
What is the difference between an independent and a dependent variable?
“variables thought to influence changes in another variable (the dependent variable).”
Known as the IV (sometimes called explanatory variable or PREDICTOR variable in non-experimental research)
When more than two independent variables are used in a “factorial design” the IVs are referred to as factors.
“variables are thought to be changed by another variable (the independent variable).”
Know as the DV (sometimes called outcome variable or CRITERION variable in non-experimental research)
What is the difference between a research question and a research hypothesis?
“a formal question posed to guide research.”
The explicit questions researchers ask about the variables that interest them.
When researchers feel confident enough to make a prediction, they advance a hypothesis
“a tentative statement about the relationship between independent and dependent variable(s).”
Descriptive stats will be covered in Weeks 6-8!
The statistic that you ultimately employ should correspond with the RQ/Hypothesis:
Internal validity is the extent to which we can infer that that IV caused the change in the DV.
Internal validity depends on the strength or soundness of the design and influences whether one can conclude that the independent variable or intervention caused the dependent variable to change
Randomized experiments are the most likely to provide evidence that the independent variable caused changes in the dependent variable.
Randomizing individuals into groups maximizes the likelihood that some shared characteristic or attribute is not influencing the IV-DV relationship.
In order to infer that one variable caused a change in another:
In purely descriptive studies there is no independent variable and no attempt to find relationships (causal or otherwise) between variables so the issue of causal relationships is not relevant.
The quasi-experimental, comparative, and associational approaches all attempt to identify relationships between variables. In published studies, these three approaches usually satisfy the second criterion; the independent variable is related to the dependent variables.
The first criterion, the independent variable precedes the dependent variable, also is often met, especially in quasi-experiments.
The third criterion for causation, ruling out other possible explanations, is never possible in the comparative and associational approaches and is problematic in all but the strongest quasi-experiments.
TL;DR: If you aren’t randomly assigning your participants to conditions, then you have a weak argument for causation.
Internal validity is evaluated based on:
Are the groups equivalent in all aspects prior to the induction of the IV?
We ensure this through random assignment
If people cannot be randomly assigned to groups, we can assign the treatment randomly to the people. For ex: assigning classes to specific conditions
Groups being compared by attributes are seldom equivalent to one another.
Can incorporate methods after the study to show some effort at internal validity.
There is only one group! Nothing can demonstrate causation.
What does random assignment to groups not eliminate?
Do participants learn information about a study as they go?
Are there environmental conditions that might influence participation?
A good study should be high in both areas!
If not, must be cautious about saying that IVs influenced, impacted, or caused DV to change.
Measurement is often reliable. Participants who score low on a measure may score higher (closer to the mean) the next time.
Participants drop out for a number of reasons, but it influences group composition.
Problems are created when groups are assigned based on similarities - not randomness.
What is sampling?
Sampling is the process of selecting part of a larger group of participants with the intent of generalizing from the sample (the smaller group) to the population (the larger group).
External validity is the extent to which samples, settings, and variables can be generalized beyond the study.
All the participants of theoretical interest to the research and to which he or she would like to generalize.
Also called sampling frame
The group of participants you actually have access to, perhaps through a list or directory.
The smaller group of participants selected from the larger accessible population by the researcher and asked to participate in the study.
The participants that complete the study and whose data are actually used in the data analysis and in the report of the study’s results.
The goal is for our sample to be representative of the target theoretical population.
This maximizes external validity
Everyone has a known, nonzero change of being chosen
No method to estimate probability of being included. Sampling bias automatically becomes an issue.
All involve randomization at some point!
YES! Very much so.
Larger samples will identify effects that smaller samples will not.
The general rule of thumb is that representativeness is more important than sample size.
A huge sample that is not representative will still be misleading.
Can calculate power to determine an appropriate size.
A good study should be high in external validity!
If it isn’t, must be cautious about generalizing the results to the larger population.
Can you identify each key area in the study?
Whether the conditions, settings, times, testers, or procedures are rep of natural conditions and so forth and, thus, whether results can be generalized to real life outcomes.
AKA is the research environment similar to the natural environment? Does the manipulation of the IV feel real to the participants?
Random selection is important for high external validity.
Random assignment is important for high internal validity.
Let’s say your group wants to study men and womens’ reactions to violent crime shows on national TV.
Quant methods are not always about getting it right, but about defending your choices.
Next week will focus primarily on manipulated independent variables in the form of experimental and quasi-experimental designs. We will also cover issues when conducting survey or other types of observational research and when independent variables differ across perspectives.